Papers by Joseph J Peper
MDBench: A Synthetic Multi-Document Reasoning Benchmark Generated with Knowledge Guidance (2025.findings-acl)
Copied to clipboard
| Challenge: | Multi-document reasoning is an area of increasing relevance given LLM capabilities in handling longer-context inputs, but few benchmarks exist to rigorously examine model behavior in this setting. |
| Approach: | They propose a new dataset for evaluating LLMs on the task of multi-document reasoning that uses condensed structured seed knowledge to modify it through LLM-assisted edits. |
| Outcome: | The proposed method generates document sets and QA examples on a multi-document reasoning task using a synthetic generation process. |